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1. Introduction 



1.1 Background 



Throughout the 1980s, the competence of the UK workforce 
became an increasingly important concern. Comparisons of the 
quality of our workforce with that of our international 
competitors established that, broadly speaking, our workforce 
were less productive than our competitors and that this 
undermined our ability to compete. Nevertheless, the common 
perception that UK employers 'do not train' is a misconception. 
The latest figures available show that in 1990 a total of £33 
billion was spent on training in Britain'. The debate has shifted 
away from being solely whether or not we train enough as a 
country, to also cover whether the money is being spent wisely. 
Increasingly, the concern is that employers get the best returns 
on their investments. 

Despite this, there is still apparently little systematic attempt to 
ensure that spending on training is money well spent. Various 
surveys have found that evaluation of training is, in the majority 
of firms, fairly rudimentary and that where it does exist, it tends 
to focus on costs of training rather than actual benefits. A recent 
survey (Training Trent/s, Industrial Society, 1993) shows that a 
only quarter of respondents are satisfied with the methods being 
used to evaluate training and development, with nearly half 
reporting some levels of dissatisfaction. This results in a rather 
vague perception of the value of training programmes and a 
reliance on 'acts of faith' rather than a clear analysis. This can 
lead ultimately to a reluctance to devote resources in this 
direction. As the Training Trends (op cit.) sur\'cy notes: 

Most or^iwisatiom either fail alto)^cther to evaluate the effectiveness 
of the trainin'^’ thei/ pay for, or believe that the ehecks that they do 
earry out are. at least, badly flawed. . . . Nor is the situation even 
improving.' 

Ironicallv, there is an extensive literature on training evaluation, 
ranging from practical and prescriptive texts to complex 
theoretical models. What is clear, however, is that there is more 
being written (and said) about measuring training effectiveness 
than is actuallv taking place in the workplace. In these 
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circumstances, training and development does become such an 
act of faith, tolerated when money is available and when there is 
an immediate perceived need. For training to become integral to 
the workplace, evaluation has to move beyond this and 

demonstrate that investment in training is linked to business 
success, able to contribute to business objectives and capable of 
making a return for the organisation. 

There are signs that there is a changing mood in the UK with 
regard to measuring the effectiveness of training. Investors in 
People has as one its four criteria that an employer must 
'evaluate the company's investment in training and 

development to assess achievement and to improve future 
effectiveness'. At the time of writing, over 900 employers in the 
UK had achieved the Investors standard and about 9,000 had 
made a public commitment that they would work towards doing 
so. A recent Industrial Society publication (Managing Best 
Practice, 1994) notes that there is now an almost complete 

consensus on the importance of training evaluation, that 

commitment to training evaluation has risen markedly in the last 
two years arid is set to rise sharply over the next two. They 
suggest that there are three key factors behind this move: (i) 
business efficiency considerations, as economic pressures make 
it increasingly vital for employers to tighten up on how the cash 
is being spent, (ii) the impact of Investors in People and (iii) 
internal pressures from the organisation's own senior managers 
and trainers. 



1 .2 Report structure 



The report is based on a synthesis of the literature on training 

evaluation, combined with evidence from employers who are 

undertaking some evaluation of their training. 

The report is structured in the following way: 

• Chapter 2 contains a brief discussion on the rationale for 
undertaking an evaluation of training. 

• Chapter 3 outlines the overall concept of the training system 
which has been developed in the theory over the last two 
decades and outlines the model that will be developed 
throughout the remainder of the text. 

• 1. hapters 4 ti> 9 expand upon each of the stages described in 
this model to give a clear description of actions that are 
required in each of these steps. 

• Chapter 10 examines how the co.sts and benefits of the 
evaluation should be calculated and how they should be 
presented within the organisaticm. 

• Chapter 1 1 presents our c('nclusions and summary. 
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2. Rationale and Process of Measuring the Effectiveness 
of Training 



To mensurt' the e'fectiveness of tiaining can be a difficult and 
costly thing to do. Before attempting to set up structures to 
allow such a measurement to take place, it is as well to be sure 
that such an activity is worthwhile. The question of 'why bother' 
can be addressed at two levels: why bother at all and why 
bother in particular circumstances.' (ie how to prioritise the 
evaluation activities.) 

2.1 Why bother to evaluate? 

Newby gives the most straightfonvard answer to this 

question: that constructive, practical evaluation of training is 
available to anyone with a serious interest in training and that 
the benefits of doing so substantially outweigh the costs. Newbv 
identifies six direct benefits: 

• Quality control: quality control systems are designed to 
ensure that products or serx’ices are fit for their intended 
purpose. Evaluation in training will assess the extent to which 
work-related results can be demonstrated to arise from the 
training. Successful, positive elements of training can be 
maintained and reinforced, whilst negative elements removed 
or revised. If results cannot be justified, then it becomes hard 
to justify the commitment of any resources to the training 
activity and they can be re-allocated to where they may make 
a greater impact. 

• Efficient training design: it throws an emphasis on those 
elements of a training system which matter, su 'h as a proper 
definition of objectives and setting criteria on how these 
t>bjeclives are tt> be measured. 

• Enhanced professional esteem: training professionals can 
gain enhanced stature from having systematic evaluation 
data rather than intuitive assessments of their contribution to 
the business. Being assessed on their contribution to the 
'bi'iiom-line' ih the business puts the HR function on the 
same looting as other functions, instead of claiming that the 
nature of their work does not alhnv an application of the 
same criteria. This may help to break down the barriers facing 
the integration of 1 IK professionals within organisations. 
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• Enhanced negotiating power: on much the same tack, 
evaluation makes it possible for the HR function to 
demonstrate a successful contribution to the business over a 
period of time. When resources are to be allocated and new 
investment decisions to be made, then being able to show the 
outcomes of training could be invaluable. 

• Appropriate criteria of assessment; individuals within an 
organisation will make judgements about how effective the 
training function is, regardless of whether an evaluation 
system is in place or not. Given this, it is very important that 
the HR controls the choice of appropriate criteria, which it 
can most safely do on the back of a formal evaluation process. 

• Intervention strategy: evaluation can be a tool for changing 
the way that training is integrated into the organisation. It 
offers a means by which the HR function can build on its 
enhanced esteem and negotiating power to play a more active 
role in developing policy. 

2.2 Prioritising evaluation activities 

The second of these issues is perhaps not whether to evaluate or 
not at all, but which training activities to evaluate. Evaluation 
can be a time consuming process, and time spent on evaluation 
is time that cannot be spent elsewhere. Trying to measure the 
effectiveness of every training event is probably impossible: even 
if they could all be identified, then it would be impossible to 
encompass all of them within an evaluation framework, even 
given an unlimited budget. In trying to set up systems for 
measuring effectiveness of training, a decision on priorities will 
be necessary. Again, Newby (1992) offers some guidelines: 

• Importance: what could go wrong if the training is not 
successful? In some senses, it could be life threatening {eg the 
loss of life for inadecjuate safety training), in others it could 
have a serious impact on the business capacity, particularly 
where the activity is vital in achieving business objectives. 

• Frequency: how regular is the training activity? If it is a one- 
off, not to be repeated, then evaluation will not generate any 
information to allow future modification (although it may 
allow lessons for similar training). If, however, the training is 
a regular event (eg an induction programme) then evaluation 
becomes more worthwhile as information can flow back to 
improve its design. 

• Cost: how costly is both the training event and the 

evaluation? If the training is a major item in the training 
features, and accounts for a significant amount of the training 
budget, then it should be evaluated. If nothing else, such 
significant items are normally obvious targets for 'trimming 
when times are hard and it is as well to have a proper defence 
prepared. In terms of the cost of the evaluation, the more 
information required usually means more expense in terms of 
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^.ithering and interpreting it. However, despite the expense, 
the evaluator should always endeavour to get the most 
appropriate information for the evaluation. 

Another combination of factors which may add a dimension to 
this consideration, relates not specifically to the training event 
but to the sphere of performance that the training is designed to 
improve. These factors relate also to the prioritisation of training 
delivery (and we refer further to this in Chapter 4). The factors 
that need to be considered are: 

• task importance; how important is the task to the 
organisation? 

• task difficulty; is the task simple and easy to master, or is it 
complex requiring the development of sophisticated skills? 

• task frequency: how often does the task take place? 

It is perhaps best to consider these at the same time. Below is a 
diagram of this three-dimensional problem. 

Diagram 2:1 Three dimensions of training 




Task frequency 



The shaded square in the top left of the matrix shows an activity 
that would appear to be a priority for both training and 
evaluation, as it relates to a task that is difficult, performed 
frequently and is very important. At the opposite part of the 
matrix, tasks that are not difficult, not performed regularly or 
important should not be a priority for either training or 
evaluation. lasks which fall between these two ranges will 
require a more qualitative assessment: the grey square shows a 
task that is not performed frequently, but is both important and 
difficult (for example, a business strategy review), which would 
results in long-lasting consequences. 
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Newby (op cit.) suggests that evaluators should ask themselves 
what effect the evaluation will have on the organisation. If the 
answer is none, then it must be questioned whether it is 
worthwhile doing the evaluation; it may be more worthwhile 
exploring why the evaluation will have no impact. Two 
scenarios are possible: 

• Where the evaluation shows that no change is necessary. This 
does not mean that it has not been worthwhile: it may not 
have led to any change of behaviour, but confirm a widely 
held (but unvalidated) view that a training programme 
making an effective contribution is worthwhile, even if it 
draws the 'we told you so' type of reaction. 

• Where the evaluation shows that a change is necessary, but 
that the results are ignored. The reasons for this will be 
bound up with organisational factors so complex that they 
cannot be covered in a text such as this, but suffice to say an 
evaluator always needs to be sensitive to the politics of the 
organisation in which they are working. 

Where evaluation might not be appropriate 

As evaluation is an expensive and time-consuming process, it is 
worth considering whether a realititic evaluation can be achieved. 
The two things to consider are whether: 

• the timing is appropriate, where although a decision to 
abandon an evaluation would be rare, activities may be 
postponed because the timing is not appropriate. Reasons for 
inappropriate timing may include: 

• The training may have taken place too recently or too 
distantly to allow a realistic evaluation. In practice, a too 
recently completed training event should not stop all 
evaluation activities, allowing interim measures whilst 
longer term effects are allowed to develop. If the training 
was too distant and nothing has been done, then it may 
be too late for remedial action and it may be more 
profitable to focus on up-coming training events rather 
than older ones. Ideally, evaluation should be planned to 
take place when it is expected to contribute. 

• Other events preclude evaluation activities, such as any 
i*ther major change in working practices. In these cases, 
the most appropriate question to ask is probably whether 
the training should have gone ahead in any case. 

• the cost is unteasonable, particularly as a proportion of the 
inerall spend on the training event. If the cost is 
unreasonable, then an alternative evaluation methodology 
should be considered. Hstimates of what are reasonable costs 
vary but, as a guide, it is perhaps worth noting that 
tiovernment Departments consider that about one per cent of 
a training programme's total training budget should be spent 
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on ovaluiition. Of course, the proportion of money spent on 
evaluation activities will vary through the life of a particular 
training event, with disproportionate amounts being spent at 
critical periods (eg training development) and less whilst the 
training programme is up and running. 

2.3 Who does the evaluation? 

A feature of the late 1980s and early 1990s has been the 
increasing involvement of tine managers in the delivery of HR. 
These changes have largely mirrored the trend of pushing down 
accountability for running business units or delivering services 
to the lowest possible level of the organisation. Bevan and 
Hayday (1994) examined this changing responsibility and how 
they are supported in the rote. The evidence found that the 
structure of the organisation often affected the rationale for such 
a development: those which are targe, diversified or 
geographically spread arc more likely to have decentralised 
personnel functions. In alt the case studies of this study, 
evidence was found of tensions between the HR function and 
line managers, which limited the real extent of developed 
accountability achieved. However, it is argued that by passing 
the more routine functions down to line managers, the HR 
function could concentrate on developing its role as a centre of 
excellence, allowing staff to act in a consultancy role, 
considering strategy and leaving them available to advise on the 
more complex problems of personal management. 

This development has obvious implications for measuring the 
effectiveness of training and a number of issues need to be 
addressed: 

• Should measuring the effectiveness of training be a subject 
that is devolved to line managers? The answers probably lie 
somewhere between the two: it is an issue of obvious 
strategic importance and yet most of the inputs have to derive 
from employees themselves and their line managers. 

• Ciiven that evaluation of training was notoriously weak 
before devolution, it can be argued that it is singularly 
inappropriate that HR personnel now see tit to advise line 
managers on how to approach this. 

• What level of support can HR managers offer to tine 
managei^. Appropriate materials need to be developed 
which can make these tasks simple and routine, and yet are 
sciphisticated enough to acldress the complex issues we have 
identified in the earlier chapters. 

1 here is perhaps also a reverse issue: if the 1 IR function expects 
the tine managers to adopt some of the HR roles and become 
more like personal managers of old, is it not unreasonable to 
expect that HR managers should become more tike business 
managers. 1 llicU ( 1989 ) argues that there does not appear to be 
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An adequate language to describe the link between business 
strategy and training: whilst accountants have a language that 
will make their role understandable to the many managers who 
are not accountants, personnel managers do not share a common 
language or even the same concepts. Jackson (1989) concurs with 
this, arguing that trainers should consider themselves as 
'business man or woman' and re-define their terms to those 
more widely understood elsewhere. Jackson notes that 'business 
results are typically measured in financial terms: this is the 
currency of the business man or woman. This is the currency of 
the trainer.' 

The devolution of HR functions to the line manager is a 
relatively new phenomenon, but one that appears set to continue 
and place new responsibilities on individuals who previously 
did not hold these. Training Trends (1993) confirms this. It notes 
that in most organisations (79 per cent), responsibility for 
training evaluation lies with the individual's line manager, with 
about half of the respondents stating the training department 
also has a role to play. Training Trends notes that: 

‘These findings confirm the incrcasingli/ vital role that line managers 
now plan in the training and development of staff. With growing 
pressure on organisations to get the maximum value from training, it 
IS clearly important that the line manager's evaluation role is an 
active one.' 

A more sceptical note is provided by the Industrial Society 
(1994), which notes the comments of its specialist consultant in 
training evaluation: 

I was taken aback by the results of the Training Trends survey 
showing that three-cjuarters of evaluation is handled by line managers. 

I simply do not believe that. The eases of evaluation that I know are 
almost all carried out by the training department with some 
involvement by line managers, although usually not enough 
involvement.' 

Bramley (1991) notes that if training is fully integrated into the 
organisation, the role of the training manager becomes much 
more wide-ranging: essentially that of an internal consultant. 
However, this may create some problems: this shift would call 
for a new set of skills on behalf of the training manager. Not all 
may welcome this, and not all will have the skills. 

2.4 Considering the outcomes at the outset 

The question of why the e\ aluation is taking place may seem to 
be semantic. However, the question needs to be asked by the 
evaluator each time an evaluation takes place because it can lead 
to the structuring of the eva'uation and to ensure that the 
exercise produces meaningful results. As noted by basterby- 
Smith and Mackness (1942): 
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'Most evaluations are a waste of time unless they start by clarifying 
the purposes they are intended to serve, and this usually means 
clarifying whose purposes are being served.' 

The fact that there are different groups, with different interests 
in the outcomes of the evaluation should be explicitly 
considered. Easterby-Smith and Mackness name some of the 
typical stakeholders (Table 1:1 below): perhaps most 

organisations would add budget holders and line managers to 
this list. 

Table 1:1 Stakeholders' requirements of the evaluation 



Stakeholders 


Interest 


Dominant purpose 


Sponsors 


Mooting deadlines. 

C'oiiif)arison with other regions. 


Proving that training 
is enabling effective 
implementation. 


Dosigtu'rs 


Verifying that material is 
having the desired impac t. 


f^roving that materials 
work. Monitoring and 
controlling the use of 
material. 


'\ rtiiJUM's 


liic*ntifying and ( correcting 
vv(*ak areas in course's. 


Improving the overall 
equality of training and 
their own performance. 


Train(M‘s 


1 nsuring that they have 
learnt what will be needc'd. 


learning what will be 
required lo operate 
effectively. 
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2.5 Summary 



before any activity is undertaken, the evaluator needs to 
consider three (related) ejuestions: 

• why is the evaluation taking place? 

• who will use the results of the evaluation? 

• what will the results be used for? 

The answers will vary from case to case, and there may even be 
more than one answer for each of the questions. However, if 
these questions can be answered, the evaluation will have a 
statement of purpose, which will allow the evaluator to 
progress. 
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3. The Training Cycle 



3.1 Introduction 



This chapter oiuhnes an overall framework within which the 
measurement of effectiveness of training may sit. The literature 
calls this framework many things: a training framework or a 
systems approach are but two of the more common phrases. 

Regardless of the title, the overriding message is that training 
should not be considered as a static event but as a process where 
information on the effectiveness of training is a natural and 
essential part of the overall training process. It is useful to think 
of this as a cyclical process by which such information is fed 
back into the design of the training programmes which follow, 
l or this reason we have called it a training cycle. 

Whilst this may appear to be essentially 'old hat' it remains 
striking how little it is applied in practice. The Industrial Society 
(1994) asked whether organisations followed any particular 
route: only a tenth used any systematic method (eg a cyclical 
approach d la Kirkpatrick), with a fifth saying that they used no 
systematic evaluation method, and with 56 per cent saying that 
they did not know. These responses, it should be remembered, 
came from the same organisations that reported in Chapter 1 
above, that evaluation was of the greatest importance and with 
importance increasing all the time. 

3.2 The training cycle 

A training cycle consists of a series of steps which lead to a 
training event being undertaken: evaluation provides feedback 
which links back to the initial stages of training design. Indeed, 
it is the evaluation/ feedback process which makes this a cyclical 
event: without it training would be a linear process leading from 
initiating training through to its implementation. 

This kind of approach is not new and is not unique to the field of 
training and development. Similar systems approaches have 
been fashionable in control engineering and cybernetics since the 
1950s. More recently has been applied to economic, industrial 
and sociological systems. 
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The Armed Services have been delivering training in this fashion since the 

early 1970s. Their view of the training cycle, which they term the Systems 

Approach to Training (SAT), is that of: 

• A training philosophy which emphasises the inter-relationship between 
training and other systems, such as personnel management, supply and 
logistics, and finance — and the inter-dependence of the component 
parts of a training system. In applying SAT, training is undertaken on a 
planned basis In a logical series of steps. . . . Fundamental to the 
f)hilosophy is that these steps constitute a cycle with the evaluation 
bringing about a reassc'ssment of needs and a consequent refinement of 
the training given. 

It emphasises three imf)ortant elements: 

• that training comf)hses a senes of logical steps 

• that these steps form a cyck', with constant evaluation and re-appraisal 

• that training f)rovision should l)e seen in the context of the wider 
organisation. 

(taken from Spilsbury et a/., 1992) 



What this model docs do is emphasise the point that to measure 
the effectiveness of training is rarely something that can be done 
as a discrete activity after the training event has taken place: 
more often than not, the necessary information will simply not 
be available. Evaluation has to be considered before the training 
event takes place and in most cases, evaluation will be most 
effective when it is part of a system which examines the entire 
training process. Within this, the manager will still need to use 
appropriate tools to be able to measure the impact. 

Below, we describe a model which has been synthesised from 
various different types of cycle (see, for example, Heame, 1985; 
Kirkpatrick, 1983; Newby, 1992) which we believe takes account 
of all the major steps. 1 he steps in this cycle arc: 

• Stage 1: identification of training need: examining what skills 
and attributes are necessary for the job to be undertaken, the 
skills and attribute* of the job holder and the extent of the 

• Stage 2: design, preparation and delivery of training. 

• Stage 3: discovering the trainee's attitude to training 
(reaction) and whether the training has been learnt (learning). 
Reaction involves the participant's feelings towards the 
training process, including the training content, the trainer 
and the training methods used. Learning is the extent to 
which the content of the learning event has actually been 
absorbed by the trainee. 

• Stage 4: discovering whether the lessons learnt during 
training have been transferred to the jt)b and are being used 
effectivelv in doing the ji>b. 
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Diagram 3:1 A training cycle 




• Stage 5; evaluating the effects of the training on the 
organisation. This is the area in which there is perhaps most 
confusion, and subsequently little real action in the 
workplace. 

• Stage 6: reinforcement of positive behaviour. It is optimal that 
any positive outcomes are maintained for as long as possible. 

It is not a rare event for changes in behaviour to be 
temporary, with a gentle slide back to previous ways of 
working. 

It is important to note the feedback loops. Feedback on the 
process of actually delivering the training can come from the 
reaction and learning stage, the transfer of the training to the 
workplace and the evaluation of the impact of the training. The 
main feedback for the identification of training needs comes 
from an assessment of the transfer of work to the training and 
the evaluation of the impact on the organisation. 

It is important to note that these feedback loops may consist of 
two very different types of information. Bramley (1990) notes 
that an evaluation can ser\^e at least two purposes: 

• to determine the worth of training to the organisation — a 
process best done bv quantitative methods and with hard, 
numerical data 

• allowing insights into the methods of learning, where the 
experiences of those involved are the main focus, thus using 
mainlv softer, qualitative information. 

Ihere is dbvioiislv '’alue in both of these positions. Bramley 
implies that the simultaneous satisfaction of both is difficult to 
achieve. This need not be the case, and perhaps a useful 
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distinction is one sometimes made between evaluation and 
\ alidation, whereby: 

• validation is that process which ensures that training meets 
its design criteria and objectives, and 

• evaluation is about providing measurable outcomes from 
training which can be used to assess the impact on the 
workplace performance of participants. 

In our view, a system to measure the effectiveness of training 
which is a useful tool for management has to fulfil both of these 
criteria. A system which tells you that there has been no added 
\cilue as a result of training, but which does not tell you why, 
has not progressed the organisation very far. Similarly, a system 
which informs us that the training has met its design criteria, but 
does not let us know whether this had made any impact on the 
organisation, could lead to general feelings of frustration on any 
other than training professionals. 

The system we outline below may be used to deliver 
information of both types. However, the point is well-made that 
different evaluation techniques can lead to very different 
outcomes, and the evaluator should know what they want out of 
the evaluation before setting out. In general terms of the 
diagram above, the feedback loops which flow into the design, 
preparation and delivery stage are validation loops, those which 
flow into the identification of training need stage are evaluation 
loops. 



The trainee viewpoint 

The above explores the view from the organisation's perspective. 
It may be as well to realise that the individuals who are to 
participate in the training and development will see the process 
from a different perspective. Trainees have needs, views on 
those needs, and perceptions of the situations in which they find 
themselves. I'or individuals the process becomes: 

• selection for training 

• briefing and preparation 

• becoming committed to learning 

• learning 

• preparing lor transfer of learning back to the job 

• returning to work 

• transfer of training and improved efficiency 

• maintenance of improved efficiency. 

It is important for the organisation that the trainee knows that 
then' exists a svstem for evaluating training. This is important 
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for two reasons: it emphasises to the trainee how important and 
seriously the organisation takes its investment in training: it is 
not a holiday, a few days off or a reward for previous good 
performance, but an investment in which the company is 
expecting a return. Secondly, a correct definition and statement 
of needs may put in place proper communication and forestall 
(or at least act as a counterweight to) unofficial communication 
down the internal 'grapevine'. 



3.3 Summary 

The training cycle as described above is not particularly new or 
novel, but we should note that: 

• it is a robust way forward, and 

• it is little used. 

The point of describing it in the level of detail above is that it is 
the overall system that will be developed upon in the remainder 
of this text. In the next few chapters we examine each of these 
steps further. 
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4. Identifying Training Needs 



4.1 Introduction 



Identifying the existence and extent of the training need is 
perhaps the most important element in measuring the 
effectiveness of training. The realisation of this is not new: 
writing nearly 40 years ago on general principles of industrial 
training, Wellens (1955) noted that: 

l^riuciplc 1: No training project should be undertaken unless the 
purpose has been clearli/ defined. . . . This principle may appear trite, 
hut if it were ri^^orously applied it would result in the disappearance of 
fuuch so-called trainhi^^at prcsetit undertaken/ 

One wonders how much has changed. Certainly we now have 
the tools to more rigorously determine training needs, but 
whether the collective will yet exists is somewhat more in doubt. 
Indeed, Mcdonald (1991) writes: 

'Despite the amount of attention which is ifiven to the term 'training 
needs analysis' one still observes department hea is waiting fo be told 
what trainini^ proi^rammes arc available for the forthcoynin^ year. . . . 
rath'r than dcsi^nin^^ training to suit ori^anisatumal or individu I 
problems, wc often work in reverse. That is, with a tempting array of 
cxistini^ proiframmes, xve then look around for the problems or 
individuals to fit them.' 

4.2 Identifying training needs 

The skill needs of the organisation need to be examined from 
two directions simultaneously: top down and bottom up. From 
top down we need to examine the new skills that the 
organisation requires, from bottom up the accumulation of skills 
from the individual jobs. 

The analysis of training needs involves a three stage process: 
examining what skills and attributes are necessary for the job to 
be undertaken, examining the skills and attributes of the job 
holder and measuring the gap. 

Ideally the organisation's skill needs should be addressed first, 
then the job analysis can be completed, with the individual's 
training need as the final component. 



C'i 
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4.2.1 Organisational context 

Traditionally, training has focused on improving the skills of 
individual workers with little attention being paid to the 
organisational context in which training takes phi^-e. Training 
should be devised in order to fit in with the overall aims of the 
organisation if time and money are not to be wasted. It should 
be remembered that training exists to serve the organisation, not 
the individuals who work within it: as Wellens (op cit.) again 
notes; 

'Principle V: Trainin;i /.s not the prime concern of industry. . . . 
huiustriiil trainiii;^ cannot be judged by the same criteria as that of a 
schoolmaster or lecturer. It must be judged according to the way in 
which it furthers the primary purpose of industry.' 

An individual may attend an excellent training course but if this 
is at odds with organisational goals, the training will either not 
bo implemented or will hinder smooth progress towards these 
goals. 

It is clear from this that aiiy training plan must be constructed 
which derives from and is clearly linked with the organisation's 
business plan so that training needs are related to organisational 
goals. Hussey (1985) found that only a third of British companies 
implemented such a policy. The situation may be improving: 
more recently the Employment Department's survey. Skill Needs 
in Britain 1994, shows that over half of e.stablishments claimed to 
have both formal business and manpower plans. However, the 
survey does not indicate how thorough these plans are, or 
whether one derives from the other. Dench (1993) found that 67 
per cent of workplaces in the HMSPS' survey had a business 
plan and 54 per cent a training plan. Where both of these existed, 
the nature of the link varied from being completely independent 
(12 per cent) to being formally linked (35 per cent). This has now 
been given greater prominence by the stress laid within the 
Investors in People programme of linking training to business 
objectives. However, a recent survey by Training Trends (1993) 
notes that most of the respondents do not know what the 
relationship is between the training plan and the overall 
business strategy. The situation seems to be one of vagueness 
rather than separation (only six per cent say that training plans 
are not formally linked to the business strategy). Perhaps this is 
indicative of the fact that we all know that the two should be 
linked, but that in practice they rarely are. 

lackson (l<^K'-t) emphasises the importance of discovering the 
corporate objectives and gaining an understanding of how these 
will impact upi>n corporate needs. In larger organisations, 
discussions with the architects of the a>mpany's mission and 
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central business plans may bo unlikely to happen. However, it is 
worthwhile setting up a system whereby a dialogue can be 
maintained between operational directors and HR directors so 
that the personnel and training implications of any new 
developments can be put on the agenda. This has a number of 
advantages: it enables an early identification of what training 
and development may be needed in the future and it brings HR 
closer to the business decision-making process, from which it is 
often excluded until the last moment. 

4.2.2 Job analysis 

Job analysis identifies the duties which comprise the overall job. 
There are several methods for doing this including job 
observation or having an individual actually do the job for a 
short period (see, for example. Ream and Kandola, 1993). 
However, possibly the easiest method is for the job holder and 
the job supervisor to describe the actual job and the tasks 
involved. Tasks are drawn together in a scalar fashion, with a 
hierarchy of levels as shown in the diagram below. Weighting 
has to be applied to their frequency and their difficulty to decide 
on the relative importance of each job task. The three- 
dimensional matrix outlined in Chapter 2 (task importance, 
frequency of performance and difficulty) is also applicable at 
this stage. 

1 lowever the job analysis is conducted, it is important that it 
remains flexible and changes as the nature of the organisation 
changes. The nature of the job may change as the employee 
performs routine tasks. More significant, perhaps, will be 'top 
down changes', where a change in the organisation's approach 
will necessitate changes in the jobs done by individuals. 

The outcome from the job analysis should be drawn together in 
the job description, which is a description of the tasks that 
individuals will be required to undertake in the performance of 
their jobs. It should be noted that at this stage it is an 
abstraction, m>t related to any particular individual’s ability to 

Diagram 4:1 Scalar job hierarchy 
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do the job, but to a range of activities that are required to be 
completed and a statement of the lev»'l at which these activities 
need to be done. 

A final point on conducting a job analysis is communicating 
with staff and staff representatives. Trade unions or staff 
associations will naturally have an interest in the design of jobs 
and the subsequent contribution of training. Most trade unions 
will want to contribute views to the process, and agreement 
should be reached on how this is most easily done and at what 
stage it would be most useful. A job analysis will also involve 
contacting a range of staff, particularly if it is being done afresh. 

It will probably be helpful if staff are informed about the 
analysis, why it is taking place, who is involved and the 
timescale. It is particularly important to stress the positive 
aspects of the analysis and give the necessary assurances of 
confidentiality. 

Development of competence-based job analysis 

The de\elopment of National Vor >tional Qualifications (NVQs) 
and their attendant Occupational standard for a wide range of 
jobs has gone a long way to facilitate the recognition of 
individual job tasks. These new national standards for 
vocational skills are now being introduced in most industries 
and are based on competence in a given occupation, not on 
completion of a training programme. The levels of competency 
required for a given task are clearly specified and they are 
assessed in the workplace by the line manager. The standards 
themselves have been set by employers through the Industry 
Lead Bodies for each industry. The role of the supervisor as an 
assessor of the trainee means that the effectiveness and 
appropriateness of the training programme will be much more 
directly apparent than in the past.' 

The Management Charter Initiative is an attempt to provide 
coherence and clarity in the field of standards of managerial 
performance. Again, this is an employer-led initiative with firm 
backing from the I-mployment Department. The MCI is 
cieveloping competence-based standards for all levels of 
managers, from supervisors to senior managers. The need to set 
standards in this way is a response to the relative lack of formal 
training for 'Managers in Britain, especially those over 35. This 
has led to the process by which managers are given credits for 
their previ ous experience, called Accreciitation of Prior Learning. 
The AI’L takes the form of an assessment which can then lead to 
the award of a forma! credit toward a cjualification. Levels of 
competence for supervisors and middle managers have now 
been set but the senior management competences are still being 



l or a fuller description of NA'Qs and their implementation, see I 
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developed reflecting the difficulty of assessing the more 
complex levels of managerial tasks. 



The Council for National Academic Awards has developed a 
scheme which allows companies to have their in-company 
training programmes assessed so that an academic qualification 
can be awarded to the participants. Credits are awarded for 
various parts of the company training scheme and these credits 
accumulate towards a certificate or diploma of higher education 
or an honours degree. This process of credit-rating for in- 
company courses is known as the Credit Accumulation and 
Transfer Scheme (CATS). The advantage is that it provides extra 
motivation to staff to achieve well on the company courses and 
allows achievements to be given formal recognition. The 
evaluation of company training is done by academic staff from a 
local polytechnic, university or college. 

4.2.3 individual training needs 

At this level, the comparisoii is the attributes of individuals 
against the job description (as described above) in order that 
gaps may be identified. These gaps are the individual training 
need. If a job specification has been successfully completed, 
detailing task, skill and knowledge analysis, any deficiencies 
should be apparent. However, there are a number of more 
formal methods of undertaking this analysis: 

• Measurement of attainment against pre-set performance 
criteria. This is particularly appropriate for relatively low 
skill, mechanical, repetitive tasks which can be measured 
electronically, ty scanning rates at supermarket check-out 
tills, keyboard skills. 

• I’erformance appraisal by line managers, which identifies 
strengths and weaknesses of individuals. 

• Use of assessmerU centres to identify promotion potential, 
particularly among management staff. 

Iremh (1993) notes that 62 per cent of organisations 
formally assess individual's training and development needs at 
least once a year, with a further .30 per cent also doing this but 
only for some grades. This is a general confirmation that there is 
a trend towards the use of individual performance appraisals by 
line managers and these are generally seen as being the general 
answer to the majority of problems on the training need 
analysis. 

1 iowever, a closer analysis would indicate that such 
performance appraisals are not appropriate for this particular 
task, but may be a useful means of to identify training 

needs. Mramley ( 190 ]) stresses tWi> reasons whv this is the case, 
in that: 
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Assessment centres 

The assessment centre is not a place but a technique. The objective of an 
assessment centre is to obtain the best possible indication of a person's 
actual or potential competence to perform at the target job or job level (see 
Woodruffe, 1 990). Several (reople will attend the assessment centre at one 
time and will be observed by a team of assessors. Commonly, there is a 
ratio of one assessor to two participants. 

The focus of the assessment centre is on job behaviour. Participants will be 
judged according to their performance in set exercises, such as; 

• in-tray exercises 

• analytical exercises 

• simulated meetings with customers. 

A 1989 survc'y in the UK found that one third of large companies 
(employing more than 1,000 people) had used an assessment centre in the 
past year. They are not used universally but just for selecting particular 
groups such as graduatoa or managers. 

The assessment centre is a high cost method of selection because of the 
high assessor/participant ratio. However, it is perceived by employers to 
give a verv accurate and high quality assessment. 



• a performance deficiency raised in a performance appraisal 
need not mean automatically that there is a training need, job 
situation factors may have a greater influence on job 
performance than on an individual's skill level. 

• training need is related to job performance and not to an 
individual's desire to attend a particular route of training and 
development. Training needs as defined here are job-related. 
Occasionally, a company may recognise undertaking the 
learning process as an end in itself. This may be worthwhile, 
but its separation from the immediate job-related training 
needs to be acknowledged. 

As noted above, the performance appraisal may lead to an area 
being raised, but it may not lead to an appropriate training need 
being analysed. In particular, it may not result in a training need 
that has been specified in a way that is suitable for measurement 
at a later date. In these cases consideration should be given to: 

• a further meeting to discuss only the training need. 
Performance appraisal meetings are often crowded meetings, 
with a full agenda. At a separate training needs meeting, the 
participants cm focus fully on what the training need is, how 
it manifests i elf and suitable actions. 

• analysing the situation further to identify the exact nature of 
the problem (see the text bc'x below). 

The most commonly named problem is that alluded to in the 
example above; how to identify and specify the training need in 
a manner that allows quantification for later comparison. This is 
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Identifying a performance deficiency 






The company identified, in its annual performance review, that one of their 
sales team was not performing as well as others of the same grade. This 
particular member of staff was relatively new to sales, having been 
transferred from elsewhere within the company. The actual training needs 
analvsis was deferred to a separate meeting. 

At this separate meeting, it became clear that as the individual was new to 
the post they had some difficulty with approaching clients. This tended to 
reflect Itself in 'displacement activity', whereby the individual felt more easy 
to cope with clients on the telephone rather than face-to-face. However, 
there was no available benchmark Information available to compare the 
amount of time that this individual spent on the telephone, compared to 
others in the sales tiMm. 

The agreed first step was to gather data. The individual concer ed and a 
sample of others in the sales team logged their activities, noting the 
proportion of time that they spent In face-to-face meetings, the amount of 
time spent on the telephone and the nature of the meetings/calls. The 
feeling about displacement activity was confirmed, with the 'problem' 
individual telephoning rather than visiting, and writing rather than calling. 

As a result of the analysis, the line manager was able to set targets for the 
individual to work to, whilst at the same time giving greater training on 
product knowledge to boost the individual's confidence. The impact of the 
training was noted in both measurably changed behaviour (process 
indicator) and measurable changes in sales (output measure). 



most easily aiidressed by persistent questioning: then the most 
useful word here is 'why'. When given the reason that the 
outcome is impossible to determine, simply ask 'w'hy is the 
training taking place?'. The response, in the first instance, will 
normally be suitably vague, at which point 'why' (or some other 
suitable form) is asked until a definable objective is reached. If 
no definable objective can be reached, then the reason for going 
ahead wdth the training must be queried. The technique is 
suitably simple, albeit at the expense of the questioner seeming 
initially troublesome, but it does seem to generate responses that 
are useable. The example below is a fictional example serving 
merely to indicate the principle. 



4.3 Costing the skill deficiency 

Perhaps the most common request on information is to show the 
return from training in financial terms. The timing of the request 
— usually after the training event and not based on a concrete 
training needs analysis — makes it impossible to comply. If this 
information is likely to be important, then collecting the 
appropriate information must begin at this stage. 

With the process described abo\e for identifying performance 
deficiencies, the appropriate tools are in place for estimating 
how such under performance is costing the organisation. To 
de'. ise a fin.im lal figure requires some assumption to be made, 
and as with all such exercises, some errors will creep in. 
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Diagram 4:2 The 'why?' chain 
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Analysis How many staff meetings? 

Time spent in meetings? 



Other evidence 



Is there upward appraisal? 

Do John Smith's rankings vary 
from other managers? 



Are there staff attitude sur\'evs to refer to? 



Are there exit interv iews? 



However, there is a procedure to make this estimation. 

Essentially this includes: 

• assessing the performance of the workers, identifying areas of 
shortfall 

• estimating the costs of the shortfall, including time spent by 
the individual and the line manager, based on salary costs 

• extrapolating the costs across all managers within the 
i^rganisation. 

Such an exercise was carried out by Coaley (199,?) and is 

described briefly in the text box below. 



4.4 Who does the training need analysis? 

There is a question over who actually does this task. There is not 
a single answer to this as much will depend on the organisation 
of the internal persimnel and IIK function. However, as Rae 
(1991) points out, it will be the line manager and the individual 
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Costing performance deficiences 

Coak'v (199 3) (lescril)es an exercise, undertaken for a major UK company, 
to cost the inv(»stment in management training. The first stage of this was to 
identify tht» ( ost of performaiu.e dcTiciences. 

losses mad(' l)\ th(» (Mnf)loving organisation were found to consist of four 
kinds: 

• direct costs of mistakes 

• direc t costs in order to rectify mistakes 

• indirect costs of time wastt‘d at different levels 

• indii(‘( i costs of loss of l)usiness. 

Both the* tinu‘ wasted and the loss of orders can be given a financial value, 
in the first c ase through the knowledge of salary levels. A wasted day can be 
translated into financial terms through a simple calculation. The total costs of 
poor performanc e is tlien calculated by a simple addition of these factors. 

The f)rcK:eciure took the following steps: 

• Structural intorvienvs conductc'd with line tnanagers in order to identify 
[)rol)lem situations related to average workplace performance. 

• for each situation', line managers were asked for details of costs in 
terms of cash paid out to rectify the situation, lost orders for the 
comf)an\ and time wasted. Salary levels were recorded in order tc:> 
('valuate the financ iai costs c:>f wasted time. 

• The findings were collated and analysed. Only directly staled and 
(]uantified costs were involved in the analysis: where there was a 
significant ('lenient of doubt, statements were not used for calculation, 
which means that overall organisational costs may be greater than those 
(‘stimated. 

lor th(' comfiany involvcnl, line manager financial costs amounted to 
£25,.S7f) in total (£881 per individual) and time costs at 982 hours over a 12 
montli period. It suggests wasted line manager time of three to four per cent 
annually. 



who will bo in the best position to identify training needs: they 
are on the spot, know the work and should be able to identify 
quickly and easily any problems which may have a training 
solution. However, the personnel/HR specialist still has a role to 
play, and may re;,ard the identification process as being a good 
way to involve the line manager in the training process. If the 
perscmnel/I IR specialist simply accepts the demands of the 
individual/line manager, then they are abandeming their role. 
Ideally, they should act as consultants or advisers, interpreting 
the expressed demands and prcxlucing a best solution. This may 
be a training soluticMi or it may not. 



4.5 When training needs may arise 



Kae (1^I9T) identifies a number of areas where training needs will 
arise. Nearly all of these involve some kind of change either for 
the organisation or the individual. It is wc^rth noting these since 
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an awareness of these change areas will help in recognising the 
need for a training needs analysis (particularly when there is no 
formal appraisal system). In addition, an understanding of the 
reason for the training need may give some indication of the 
learner motivation (discussed later in Chapter 6). Rae's main 
categories are: 

• A newcomer to the organisation: Rae notes that basic training 
may be necessary if the individual has no work experience. 
Occupational training will be given for those entering a new 
occupation. In practice, most employers offer, in addition, 
induction training to all new employees, which covers 
specific company information, which it cannot be assumed 
would be gained elsewhere, but is important for the 
successful performance of a job within the new organisation. 

• A change of work in an existing department: work of a 
department often changes, requiring either re-orientation of 
old skills or the acquisition of new' skills. The constant search 
for improvements in the efficiency and quality of work also 
requires new skills to be constantly developed for many 
employees. All these require a training or development 
response. 

• Improvement of a poor performer: identification is usually 
quite simple, via the normal line manager reporting 
mechanisms. 

There are tw o further areas w'hert Rae notes that a training need 
may arise: oevelopmental and u-> preparing an individual for 
promotion or progression through the organisation. This is an 
area where the model described above for measuring the 
effectiveness of training is most weak. Indeed, it has been 
argued that improvements in systems to measure the 
effectiveness of training may actually lead to a decrease in 
longer term developmental training, as the links between 
investment and benefit become difficult to identify and therefore 
show the lowest return. The logic of this argument suggests that 
longer term developmental training becomes the onus (and 
im’nership) i>f the individual rather than that of the organisation. 



What is not a training need? 

\'ewb\’ (19^^21 gives a list of reasons for training that can not be 
considered a training need. These include: 

• R & R training is st>metimes considered a perk of the job: a 
reward for good behavit>ur and a respite away from the day- 
lo-da\’ rigt>urs oi work. This is a common phenomenon, often 
typified by the most rigt>rous post-training evaluation of 'did 
\ ou enjov vourselt'. 1 raining should be enjoyable, but should 
not be the point of the exercise: if it is, then it should not be 
expected that one can demonstrate a return on the 
'irweslment'. 
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• Solution in search of a problem: training courses can gather 
their own momentum and become a fixed item in a training 
calendar, particularly if based on a once perceived need and if 
the trainers enjoy their delivery. A view needs to be kept on 
the continued relevance of the training. 

• Fads: opposite to the above is the latest craze of the month. 
Management science is particularly rife with grasping new 
ideas, sometimes without an over close questioning of what it 
really has to offer. A sharply focused assessment of training 
needs is an insurance against this faddism. 

• Need to spend budget: a less common problem in recent 
years with increased pressures on training budgets, but the 
need to spend the training budget before the year end can 
lead to training not linked to the business's real training 
needs. 

• Training as part of the employment package: often training is 
offered as part of the recruitment package. Whilst there is 
nothing explicitly wrong with this, a balance has to be struck 
between the needs of the individual and of the organisation. 
As has been noted above, training is not industry's prime 
function and should be designed to promote business 
activity. 

• Solutionism: often training is reached for when the root of the 
real problem may exist elsewhere. This often happens in 
appraisals, where instead of rigorously examining the 
situation, training can be seen as a quick fix. A special case of 
this is when the order for the training comes from individuals 
in positions of considerable authority (eg the MD). 

This is not to say that some of this may not have valid 
organisational reasons behind them: giving staff R & R, or 
spending the remains of a budget, may be an appropriate means 
of reward, and a 'training course' may therefore be an 
appropriate response. However, the reasons why this activity is 
being undertaken must be explicitly recognised and a 
contribution to the bottom-line not expected. 



4.6 Summary 



It is difficult to over-stress the importance of a correct and 
rigorous identification of training needs as a necessary first step 
in measuring the effectiveness of training. If this is not done 
correctly, then the validity of all else which follows is weakened. 
Ihe main message is to achieve clarity: analyse what skills the 
job requires (and wilt require in the future) and analyse what 
deficiencies indi\iduats have, aiming at all times for a 
c|iiantification. 
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5. Design^ Prepare and Deliver Training 



5.1 Introduction 



Once the training needs have been identified, the training 
programme can be designed. In this chapter we stress only two 
aspects of the design and delivery of training: the need to specify 
objectives and determining the nature of the provision. 

5.2 Specifying the objectives 

Although possibly stating the obvious, the preliminary step is to 
identify the objectives of the training programme before it is 
designed. At least, this would be an obvious statement if it were 
not the case that it seems to be done infrequently. Training 
I'nnni^ (1993) finds that only a third of responding organisations 
say they have a policy that objectives for each training and 
development programme are defined before the programme 
exists. Only a minority say that they do not, with the majority 
stating they do not know. 

There may be different levels of objectives, primary and lower 
level, but all the lower level objectives must build into the 
primarx’ objectix'e at some stage. 

Points to consider; 

• Minimise the number of primary objectives as far as possible. 
The more objectives exist, the greater the possibility of 
confusion. 

• Tlu' objectives must be measurable and achievable. 

• Make sure that the objectives are clear, precise and 
unambiguous. They should be understood by everyone to 
eliminate confusion. 

• Are the objectives consistent, in that the attainment of one 
will not be ccuitrarv tc' the achievement of another? 

Above all. the objectixes should be explicit and precise. As 
Newby (19U2) notes; 

like tint/ rllirr i'rgimhiiln>niil activilu: the dearer the 
jnetiiie ol a'lial t/iUi ivaut /i' iidueve, the easier it heeoines to plan the 
im'iins iiihi the ;^realer the prosprels ot aditevin;,^ siieeess Ineffeetive 
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training adopts a scattcr-i^un approach, rather than pinpointing the 
target. It is characterised bp learning objectives which use weasel- 
phrases such as ' . . . will gain an awareness of . . / and ' . . . should 
understand . . . ' Effective training designs are characterised bp 
objectives that elcarlp state what the trained person will be able to do 
when back hi the workplaec.' 

ft 

Where a systems approach is fully operational, objectives should 
be available from the initial training plan. 



5.3 Determine the nature of provision 

Structuring the training experience 

The next stage is to decide on what is the best mode for delivery 
of the training. I’here are no 'right' answers in this case and the 
correct balance must be found for each organisation. However, 
as Bramley (1991) points out, there are a number of questions 
which might usefully be asked at this stage, including those 
shown in the text box below. 



Course structure 

• On what basis has the coursi' been structured^ 

• Is there a sjtisKictorv bakince between practice, reflection and theory 
inputs 

• f i()v\ saiistactofA is llie duration of the course and the length of the 
working da\7 

• [.)o(*s tfie l)alance ot the course reflect the different degrees of 
imf)ortanc aUac1i(*d to the ()l)jectives^ 

Methods and media 

• On wfiat l)asis have tfie nu'tfiods l)een chosen^ 

• are optimal methods being used, given the characteristics of the 
learners^ 

• do methods and mi‘dia f)rovide variety and encourage learning^ 

• wfiat IS the quality and readability of hand-outs, computer-based training 
mattMiLil .ind training aids^ 

fva/iyaf/ng Training Fttcctivcncss, Bramley. 1991. 



All jobs will require the holder to have some knowledge. The 
extent and nature of knowledge required will determine in part, 
at least, the objectives (/V the content of the learning). The nature 
of the learning (/c method) will be determined by selecting the 
most appropriate method to achieve the objectives. The type of 
knowledge needed will have been defined by the identification 
of training needs pn>cess, and particularly in the job description. 
/\ ust'tul distinetioii of K*\ els of jobs is: 
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1. A basic level, handling isolated information or a few pieces 
of information. I'or example, being able to fill in a simple 
form or record sheet, or record simple numerical 
information. 

2. A higher level requiring the job holder to understand or use 
several pieces of isolated information at the same time in 
some form of procedure. 

3. A third level, requiring the job holder to analyse the 
information and act upon it. 

For a detailed discussion of such a hierarchical framework 
readers could refer to the standards published by, for example, 
ALBSU or summarised by Atkinson and Spilsbury (1993). 

These three different levels have quite different implications for 
the way training should be delivered; 

1. Isolated facts can be transferred easily by lectures, paper- 
based texts or other open-learning techniques. 

2. Procedures can also be transferred fairly easily by the same 
methods, although the use of some supervised practice is 
also usually recommended. 

3. This will need knowledge on the other two above and then 
practice in realistic situations. This can only be done in 
simulated situations or in closely supervised real work 
situations. If simulated situations are used, the criteria {ie the 
standards and conditions) must be clearly stated in the 
objectives of the learning. 

On and off-the-job training 

A particular issue may arise whether the training should be off- 
the-job or on-the-job. In an ideal situation, an individual would 
begin a job in a state of readiness. In reality this is not likely to 
be the case and some further on-the-job training v ill be 
necessary. This is not a particular problem if this is explicitly 
accounted for, taken into consideration and recorded (see the 
example from the Royal Navy below). Indeed, in some situations 
an element of on-the-job situation may be preferable, as the 
work situation cannot be replicated in a training situation. 

I lowever, if training is conducted on-the-job, the criteria for this 
should be no different than classroom-based training, ie it 
slum Id have (Objectives and measured outcomes. 

Rae (1492) considers on-the-job training to be a highly effective, 
flexible, relativelv low cost approach to training and to be an 
important development given the prevailing economic climate 
for the training professional. Whilst not the complete substitute 
for off-the-job training, it can play an important role in training 
and development of the workforce. 
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Performance/Competence 



The Royal Navy lays clown standards of job performance (an Operational 
Performance Statement, or OPS) and also statements of what an individual 
will be able to do at the end of any particular training period (a Training 
Performance Statement, or TPS). The difference between the OPS and the 
TPS is termed a Training C ategory and classed on a scale of one to five, 
such that: 

• T(!1 deterntines that 95 per cent of opt tional performance must be 
equal to the training performance: effectively that the individual must be 
able to perform their job on posting; 

• TC5 is token off-the-job training: nearly all learning is undertaken in the 
operating environment. 

All on-the-job training is logged in a 'task book', which details the task (what 
the trainee actuallv had to do), under what conditions (in dock, at sea, etc) 
and to what level of performance. The task book is an official record of 
achievcmient and has to l>e counter-signed by an appropriately qualified and 
ranked individual. 

Taken from 1 riming in the Armed Services, Spilsbury et al„ 1992. 



The Post Office Training and Development Group explicitly 
recognise the importance of some element of on-the-job training 
or coaching is necessary. They note that: 

'It IS unlikch/ that the ultimate desired standard of performance tvill be 
achieved during the traininif. The ultimate standard will only be 
achieved by the learner, puttinif into practice the new skill/knowlcd^c 
over a period of time and rcccivini^ on-^oin^ coaching from the person 
mjuirnig the improved performance (line manager)/ 

The situation is reflected diagrammatically below. 

Figure 5: 1 Part of a typical learning curve 




Time ► 
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Some consideration needs to be given to who actually conducts 
the on-the-job training and in what circumstances. As seen 
above, the Post Office believes that the appropriate model is to 
have the line manager as coach. Rae (op cit.) notes, historically, 
the image of the on-the-job trainer has been someone who has 
tended to be someone who has worked in a manual capacity and 
who has been drafted into the job with little or no training. The 
situation in practice is significantly differently from this and 
there are a number of different options available when 
providing on-the-job training: 

• GAFO; the simplest method; 'Co Away and Find Out'. At its 
crudest, the potential trainee approaches someone who is 
supposed to hold information and is told to 'GAFO'. 
However, it can also be a real learning opportunity if 
sufficient open learning support systems are in place. In these 
cases, trainees need to be taught how to learn and how to use 
the training materials, in whatever media are available. 

• Sitting with Nellie: the trainee is placed under the tuition of a 
worker (Nellie) in order that they can learn the appropriate 
skills. Often derided, it can be one of the most effective means 
of training, as there are few situations in which training by 
this method cannot be achieved. I lowever, as a method, it has 
freeuently failed in the past because the 'trainers', whilst 
having the required occupational skills, may not necessarily 
have the teaching/ tutoring skills or given the resources or 
support materials with which to pass on their expertise. The 
training may also be unstructured, (eg lacking objectives) 
perhaps even to the extent that it is not recognised as training. 

• One-to-one instruction: an extension of the above, with the 
exception that 'Nellie' is actually a qualified trainer for the 
organisation. 

• Coaching: the manager or supervisor will select work-related 
tasks for the trainee to perform, and ensure that they are 
performed correctly. This form of training should ideally be 
on-going, as a manager can constantly be seeking 
opportunities to develop staff from circumstances which 
develop within the workplace. 



5.4 Summary 



There is a great deal of literature which deals with appropriate 
design and deliverv of training and it is not the main focus of 
this report and so is dealt with only briefly. 

It mav seem a statement of the obvious, but the major issue 
again revolves around clarity, particularly the need for a 
definition t>f the objectives of anv training event. Without this, 
the link between the identification of training needs and ttie 
actual damage is weakened. 
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When measuring the offecti' eness of training is considered, the 
view is that only training events (seminars, training courses, 
workshops, etc.) are applicable. This is not necessarily always 
the most appropriate method and so we have included a brief 
section on on the-job training. With regard to evaluation, the 
same principles apply as for off-the-job training and should be 
applied with the same rigour. 
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6. Initial Testing: The Trainees' Attitude to Training and 
Whether the Training Has Been Learnt 



6.1 Introduction 



This chapter involves the initial testing, which deals with 
assessing the trainees' reaction and learning of the training, 
where; 

• reaction is the participant's feelings to the training process, 
the trainer and the training methods used. 

• learning is the extent to which the content of the training 
programme has actually been absorbed by the training. 



Most training is evaluated at the 'reaction' level by 
administering a short questionnaire, the so-called 'happy sheet', 
at the end of the training programme. Finding out how trainees 
felt about a training programme by means of a questionnaire 
aft^r the course is the most prevalent form of evaluation used by 
employers. However, it is not the only method used; Training 
Tirnd^i notes that whilst 63 per cent of employers use such 
questionnaires, a quarter also use interxTews or feedback 
sessions. 

It is important to find out these responses and to use them to 
feed back into training design but as a method of evaluation 
alone their usefulness is limited. The main purpo.se of these is to 
find out about the training experience: was the delivery 
adequate, were the conditions of the training event amenable to 
learning, cfc. 

1 lappiness sheets are miist commonly seen or used when 
trainees are sent away on an off-the-job training experience; they 
are awav from work and are given a discrete time to actually 
record their views. I'his need not be the C3S‘' and happiness 
sheets haw an equallv valid role to play in recording views 
about on-the-job training experiences — a fact often overlooked. 
As on-the-job training constitutes the majority of learning 
experiences, it is as important that the employer/trainer is aware 
of the nature and reaction to the training for these experiences as 
the more traditional reimned 'events'. 



6.2 Reaction 
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Happy sheets are often derided, perhaps unfairly. They have a 
role to play in that they can provide valuable information about 
the training experience. The disregard concerning their use has 
probably arisen because of the extent that companies use them 
in isolation. However, they should be seen for what they are and 
their misuse in the general evaluation framework should not 
detract from their immediate value. 



6.3 Learning 



If the training has to lead to changed behaviour on the part of 
the trainee, then it must have led to some skill or knowledge 
being imparted and learnt by the trainee. The training will not 
therefore transfer itself to the work environment if it has not 
been learnt: a set of new learnt knowledge tests is therefore not 
an unreasonable way forward. 

6.3.1 Testing for learning 

As we have seen, the type of knowledge that has to be 
transferred will vary hugely according to the nature of the job 
and so, therefore, will the means of finding out whether it has 
been learnt. We are fortunate here in that much of the training 
effectiveness literature is concerned with setting out techniques 
which assess this learning level, and there are many readily 
available techniques for testing whether training has been learnt. 
The precise method chosen will depend on the type of training 
it.self. Simple, repetitive skills, like using a keyboard, are easily 
measured before and after training. Similarly, training on 
specific topics, for example, product information for sales 
personnel, is relativ ely easy to test. In general, the more complex 
the training the more difficult it becomes to test learning. 
Hramlev (1S)91) describes techniques which can be used to test 
knowledge learned during a training programme and a 
summarv is given below. 

The value of such tests can be enhanced by their use in 
conjunction with pre-tests. If the trainee is tested at the outset, 
then re-tested, then the gain in knowledge can be tested. There 
are issues to consider both in undertaking the tests and in 
interpreting the scores. A tew points to note are: 

• When deciding to do this, have a viewpoint about the e:>dsting 
state of knowledge. It is a w.isted effort to test trainees, only 
to confirm that the trainees know nothing before the training 
takes place. 

• Thi' trainees will nevd reassurance before the pre-test. It can 
be verv demoralising undertaking any examination in which 
one cannot answer the questions, particularly when the event 
is i.li.'sigiu‘i.l to provi' that the trainee does not know. 
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Knowledge testing 

There are a varietv of wavs to test knowlecige depending on the type of 
knowlecigo that has l)een learnt, l iere we list a few of these techniques: 

• Open-ended questions: tlie traditional way of testing knowledge, used 
frequently in education vi<\ the essay system. Requires both regurgitation 
of facts, but also requires the student to construct a logical argument. 
This is important for some, limited, professional jobs, but is often a skill 
which IS completely irrc'levant in many jobs. 

• Short answer items: often a form of open-ended questions used to test 
knowledge of individual pieces of information. They can also be used to 
test powers of analysis. The questions start with a verb such as State, List, 
LMyel, Ca/ni/afe, Deter/n/ne, Detini\ 

• Objective test items: very suitable for testing low' levels on the 
knowledge hierarchy. Ihe trainee is asked to write only one or two 
words, or choose the correct alternative from a number offered: the 
most common form of these multiple-choice questions. Another variant 
IS tlie true false question l)ut which is only really suitable for testing trivial 
information and must l)e corrected for guesswork. 

t:ach c^f these has ifieir drawbacks. Objective test questions can be very 
easy to set, l)ut lal)orious to mark, with the ranking depending on the 
markers view|)oint and logical construct as well as the trainees. Multiple 
choice items can be marked bv semeone with no knowledge of the subject, 
but the questions are more difficult to construct. Scores must also be 
adjusted for guesswork (a candidate with no knowledge whatsoever should 
still scor(‘ about 50 per cent on a true 'false type of tc?st). 

Taken from Bramley, 1991. 



• It requires the production of two similar but different sets of 
questions and making sure that the trainees know this. 
Replicating the same test wdll result in the trainees tending to 
concentrate on those items which came up in the first test. 



• Calculating a gain ratio. The gain ratio can be calculated 
using the formula: 



Ciain ratio ~ 



Post - test score - Pre - test score , ,,,, 
Pcissible score - Pre - test score 



[o give a figure for each candidate. Bramley (1991) suggests 
that average gain ratios for a w^hole should vary from 20 per 
cent (for short lectures fcdlowed by questions) to 70 per cent 
with individual instruction on programmed packages, 
although others suggest that it is difficult to state what the 
gain ratio should bc‘ in such abstract terms. Its primary use 
mav well be to improve a training event over time, with an 
e\tensi(Mi tc^ a possible benchmarking tool for more 
sophisticated ccMupanies. 

<• 1C‘ aware oi different starting-off points. If someone had a 
good level knowledge before the training then their gain 
ratio will be ver\ Unv. I his indicates that they have not been 
particularlv stretched and the training not as valuable as it 
shmild have been certainly not as v'aluable as for those 
who started with Unv kiKuvledge and have a high gain ratio. 
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The value of the information here may be that it points to an 
inadequate training needs identification, as well as 
deficiencies in the design and delivery. 

I ho Post Office use information generated by pre- and post-test 
scores to identify the overall usefulness of the event, not as a test 
of individuals. They give three examples of pre- and post-test 
distributions. 

In a 'not much learning event', the learning event has failed to 
make any significant increase in the participants' skills or 
knowledge. 

Figure 6: 1 A 'not much learning event' 
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In tho 'everybody knows it already' event, the results suggest 
that the participants were already proficient in the subject areas 
before the training. 

Figure 6: 2 An event where everyone knows it already! 
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The ideal event is where the pre-test average scores are as far to 
the left as possible, whilst the average end-of-training scores is 
as far to the right as possible. 



Figure 6: 3 The ideal event 




.SoKur: [he Evnliiatuin Model: Ihin^ I.eoel 2 hitorniiition. I hi' I’o^t Offiee, Ithh'’ 

t'if course, the pre-test scores could be used as a screening device 
to ensure that the most appropriate participants are selected. In 
agreement with the training sponsor and the trainee's line 
manager, information could be provided to identify participants 
who have the potential to gain the most from a particular event. 
Participants could be grouped together in a way which reflects 
their existing level of skills/ knowledge, thereby avoiding wide 
variances in participants' pro '’aining levels, allowing training 
to be targeted at the appropnate level and maximising the 
learning piotential for all concerned. 

Training to achieve a change in attitudes and interpersonal 
behaviour (for example, in management training) can be tested 
using techniques such as Repertory Grid analysis, but these are 
complicated to administer. Wc do not describe these in full, but 
details of these testing methods can be found in Bramley (1991), 
Rae (1991) or Newby (1992). A brief description of the technique 
can be found in Bevan (1989). 

In general, problems with testing arise because they test what a 
trainee knows at a particular point in time. They may forget 
information fairh’ promptly but also they may have a weak 
of it. Tor these reasons, it may be more appropriate 
to test b\’ means of an aefivify (related to the work situation! 
which trainees must perform accurately. It may also be useful to 
follow up the initial questionnaire six to 12 months after the 
course, becausi- benefits may lake some time to be realised or 
alternativelv ma\’ lu' quickiv foi g,otti'n. 
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6.3.2 Reasons why learning may not have taken place 



If tests for loiirning take place and it is found that very little 
learning has taken place, then it is not unreasonable to ask why 
not. There are two general areas where the learning process 
failed: 

• failures in the delivery of the training 

• motivational problems amongst the trainee. 

Failures in the deli\ery can arise for a number of reasons: 
inadequate content, poor training materials, poor design etc. 
Motivational problems can arise because the individuals 
concerned do not have sufficient willingness to learn: learning 
takes place fastest when individuals recognise the need for 
training and are committed to it. What, for example, is the 
trainee's motivation for joining a particular training 
programme? 

More problematical from the evaluation process is to distinguish 
between the two. Once it has been established that learning has 
not taken place then what steps need to be taken? As Analoui 
(144?) points out, there is no easy way to measure commitment 
in an individual employee, but suggests that attention should be 
paid to the trainee's approach to learning, the rate of attention 
paid to learning activities, the regular attendance to training 
sessions, the degree of concentration brought to bear and the 
presentation of the output, which all provide subtle clues or 
indicators to allow an assessment of the 'will' and 'commitment' 
on the part of the trainee. Rae (1993) notes that motivation of the 
trainee will vary according to the original stimulus for the 
training need (see Chapter 3 above). In general terms, Rae 
assumes that motivation can be assumed in the case of the 
newcomer who is identified for potential progress. The same 
assumption cannot be made for training which has arisen either 
as a result of an identification of performance deficiency or 
where there is departmental or organisational change. In these 
cases, the evaluator needs to be more aware that there may be 
motivational problems in the trainee or trainees. Motivation 
could be improved if the trainee is consulted in advance about 
the training that is to be given and the reason why it is being 
gi\en. Iniiniii/^ 'IicihIs (W93) finds that, whilst 80 per cent of 
I'lnplovi’rs do the former, less than half actually tell the trainees 
how Ihi’ training fits into the overall business objectives of the 
organisation. Views on the i|ualitv of the '.raining will already 
have bei'n gatlu’ii’d in the reaction phase. 

It IS to be reim'mhi'ted tliat wi’ are talking here about gathering 
wliat is essentiallv 'soft' information on why a training event has 
failed to impart learning. Feedback will be to the training design 
and deliveiA', but little will be able to be fed on to an assessment 
of the benefit to the company. If learning has failed to take place, 
then in most cases wv can assume that this is about zero. 
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6.4 Summary 



I'inding out trainees' initial views of the training event (whether 
by the 'happy sheet' or some other method) is the most 
commonly used, but because it has been used mainly in 
isolation, it has become much derided. Discovering whether the 
training has been delivered in such a way as to make the 
learning process more effective is obviously important, but it 
needs to be seen in the context of an overall system. 

Testing to ensure that the training has been absorbed by the 
trainee is clearly an important element in measuring the 
effectiveness of training. Only if the knowledge has been learnt 
can there be an effective transfer to the job. 
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7. Transferring the Training to the Job 



7.1 Introduction 



What the company needs to know is, not only that the trainee 
has enjoyed the learning experienced and has enhanced their 
knowledge, hut whether it has improved performance in the 
workplace. The fact that the training has been learnt by the 
trainees does not mean that there will be automatic changes in 
behaviour on the job. It is quite feasible that learning can take 
place, but behaviour may not change. This chapter examines 
why such transfer may not take place, and then examines 
possible responses to the failed transfer. 



7.2 Reasons why transfer may not take place 

There are a number of reasons why new skills may not be 
applied in the workplace, including that: 

• the work situation does not support the new training 

• the trainee thinks the training progra.nme is irrelevant 

• the trainee has no motivation to apply the new skills. 

It is wise not to under-estimate the power of inertia within a 
working environment. While training may lead to improved 
performance in the classroom, these changes may not be strong 
enough to overcome resistance to change back at the workplace. 
F'or example, it has been found that individuals who are singled 
out to receive training apart from their colleagues are often 
siKialised back into old behaviour when they return to their own 
departments (Ottaway, 1986). The transfer of training does not 
occur, not because the individual does not acquire the necessary 
skills, but because of peer influence and the lack of an 
appropriate support/reinforcement mechanism. 



•\n example ot llie woik environment not supporting a training programme 
( an l>e toiind in one of our rase study companies. This company trained 
tiu’ir ( aslners .it .i (enti.il training centri' to use certain procedures to 
minimise the risk of ti.uid M.inagers <it r entr.il offic e were well aware, 
howevei. th.it these piocedures wr’re not always used hv trainees vshen 
the\ leturned to then br.inr hes tier .uise their supervisors felt they were too 
time r onsiiining the tr. lining progr.imrne itsc*lf addrc'ssed the- issue* of fraud, 
but (onsti.iints m the work org.inisation prevented thr* measures from 
being put into pi.u tir e 
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7.3 Helping transfer to take place 

It is evidently beneficial to avoid such failure of transfer and to 
encourage the transfer of training back to the workplace. 
Solutions can be designed to be training based and/or work 
based. 



7.3.1 Training based solutions 

At the training design stage, training should be related back to 
the work situation wherever possible. Similarity between the 
training situation and the job should be maximised: 
interspersing training and work situations helps the continued 
implementation of new skills. On-the-job training with a one-to- 
one mentor ensures a direct transfer. 

Training should be designed to give a wide range of experiences 
so that the principles can be applied to situations which do not 
exactly fit the training procedure. 

Transfer is especially difficult when there is a significant time 
delay between the learning i. its application: this links again 
to an appropriate identification. If the individual does not have 
need of the learning in the job then it cannot be a need. Training 
should not be given too long before the skills are actually 
needed, although the opposite case is perhaps also damaging: 
letting someone do a jt^b for a while and then training them how 
to do it. 



7.3.2 Work based solutions 

In the workplace, gtial setting is very important. At the end of 
the training event, trainees can be asked how they intend to u.se 
what has been learnt. These expected changes can be put into an 
Action Plan and a short period (six months or so) after the 
training they can be followed up to see if the plan was being put 
into effect. 

It must be ensured that what has been Lvrnt will be supported 
bv managers and supervisors in the workplace. 

One further thing that can only be done, and which links back to 
the training design, is to test the appropriateness of the 
information/knowledge learnt. When it was initially learnt there 
was no wa\ lor the individual to know whether it was useful or 
not: after a period this would become more clear. Vital 
information is available here which can be used as an input into 
training di'sign 1 opics to be coveri'd are: 

• 1 lave \ on used Ibi' inlormaticni (and if not, why not)? 

• I tow useful is the knowledge lor the job (and if not at all, why 
ikU)’ 
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Training as teams 

The' ( ()m|)any, a |)ubli( house and restaurant chain, organises its staff into 
discrete teams w»thin each f)ub/restaurant. All training (with the exception of 
Initial induction training) is given as a team where they all learn the same 
messages. The compam believe that this helps transfer because: 

O everyone is jointly aware of what they are trying to achieve with the new 
forms of l)ehaviour 

• all staff can jointly share' in the new valuers 

• commitnu'nt of all staff can be gained to the? new ways of working. 

When the training Is transferred back to the workplace, the problems of 
negative behaviour dot's not exist: the supervisors (also part of the team) are 
not operating on a different system so no conflicts arise. The compan/ 
l)elieves that of)erating in this manner gives ownership and pride from bei ig 
part of a team, whic h enai)!es commitment to be gained from much lower 
down a team structure. 



• Has there been any difficulty in applying the knowledge (and 
if yes, what was the difficulty)? 

• How easy has it been to maintain the level of knowledge, 
given reference material, etc. 



Knowledge does begin to fade over a period of time, especially if 
not re-visited. Re-visit with another post training knowledge test 
io see how much has been lost. 

A positive step that many employers have started to use is a 
spot training meeting at which a suitable action plan, building 
on the new skills can be developed. This then forms part of the 
line manager appraisal system. 



7.4 Summary 



!t is essential that tlie trainee, having learnt new knowledge and 
skills, transfers these back to the workplace. There are a number 
of reasons whv such a transfer may not take place. It should be 
ensured that these are minimised, either by work-based or 
training-based st)lutit>ns. 
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8. Evaluate the Effects of the Training on the Organisation 



8.1 Introduction 



One reason why assessing the value of training can be difficult is 
that it is hard to isolate training from other factors affecting the 
performance of a given section or work group once training has 
taken olace. For example, if employees are taught to use new 
mach aery, it may not be easy to decide whether improvements 
in peiformance are due to the equipment itself or the ability of 
employees to use it competently. Internal re-organisation and 
factors affecting the business environment may further 
complicate the issue. If only one or two employees undergo 
training within a large organisation, it could be very hard to 
measure their improved input. In fact, unless the firm is able to 
assimilate the new skills of its employees the training may be 
largely wasted. 

Theoretically, this is perhaps one of the most difficult areas to 
undertake in the process of measuring the effectiveness of 
training. Hopefully, if the preceding steps have been undertaken 
in sufficient detail then it should become easier. However, it still 
needs a consideration of the level of the evaluation (at the 
individual level or the organisational level) and the research 
tools which emplovers can use to aid them in this process. In this 
chapter, we first of all discuss the level at which the evaluation 
takes place and then the tools that can be used to isolate and 
identify any impact. 



8.2 Individual or organisational level? 

I'here is often confusion over whether the impact of training 
should be measured at an individual or organisational level. 
Most training and development activities focus on individuals, 
with the intention that the learning process will enable them to 
do their job in a more effective way. However, the objective of 
an investment in training and development is to increase 
effectiveness of parts of the organisation, and ultimately 
impriA'e llu' pi'itornianci’ I't the whole of the organisation. 

Sometimes organisaliims rest on the assumption that if the 
imlividuals within <i team have become individually more 
effective then it is a fair assumption that the whole will have 
become mi'ri' i'ttecti\e as well. Not only is this a false 
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assumptKin, but it also fails to deliver the key message that is 
required: namely that the impact on the organisation of x 
amount of training has been i/. 

This different level of emphasis implies two different emphases 
of the process. Changes in the behaviour of individuals can be 
monitored using individual appraisal, before-and-after 
measurement and other means of personalised management. 
The research tools for doing this are described below and can be 
applied. 

Although the situation is more complex, the process to measure 
change within an organisation is effectively the same as for 
individuals; identify the areas that are expected to change and 
measure them before and after the training action has taken 
place. However, this raises the issue of what organisational 
effectiveness actually is. Various types of measure have been 
classified by Cameron (1980) as being; 

• Output-directed, focusing on the outputs of the organisation. 

• Resource-acquiring, judging effectiveness by the success of 
the organisation in acquiring needed resources from the 
external environment. 

• Constituency-satisfying, where effectiveness is judged on the 
organisation's ability to respond to the needs of its various 
constituency groups. 

• Internal process-directed, with attention focusing within the 
ct>mpanv and how it operates. 

The use of all these types of measure may not be necessary in 
each organisation, and a smaller selection may suffice. The 
example shown in the text box below describes the factors that 
were considered by an organisation that was considering 
entering the Investors in People process. 



8.3 Research tools 



These approaches on their own do not answer the criticism that 
changes would have happened in any case, regardless of the 
impact of the training event. In order to be able to lake account 
of these we need [o usl* different evaluation methodologies. 

lo measure the net impact of the training programme a 
comparison must be made between what has happened 
following the programme, with what would have happened in 
the absence oi the training. Ihere are several different 
techniques w hich can be useii to di> this and beknv we discuss 
Ihi* use of: 



• control 

• matching groiif's 
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Evaluating the impact of Investors in People I 

To heconu’ iiivokod m lnu‘siors in Poopir re<)uir('s ( ommitmont of time and I 
effort \)\ the comp‘in\ It the (<)(n[)<mv is willing to do this, then it may [)C prudent I 
befort* st'tting oft down this routt' to consider how to assi'ss thc^ impact that I 
ac hit'ving th(‘ Standard will at tualiv have upon their c om[)<mv I 

Measuring the impact I 

The employer decided to measurt* tlic' impac I of Investors in tc‘rms of outcome and I 
process measures, where I 

• Outcome measures measured l>usiness outcomes. For this company engaged I 

in the service sc‘C tor it was dec i(k‘d that the main measures would be c ustomer I 
sail .fac tion and profital)iiit\ Itwels 'both as <i percentage of turnover and per I 
emplo\ t‘C‘) I 

• Process measures i‘\amin(‘d tac tors whic h it was thought that Investors would I 

direclK im[)ac 1 upon: statt motivation, staff pertormance and the training I 
s\ stem. I 

/. .sraP mopvar/on was measured in a number of ways. Indicators of staff I 
morale W(‘re created l)\ using turnc:)ver rates (excluding enforced I 

turnover sue li as redundanc les) or sic kness/dl)senti‘eism rates. The I 

com[)an\' was fortunate in having rec c'ntly conducted a staff attitude I 

suru's which was [)erhaps the most rigorous measure available. I 

J. Si.if/ pi-r/omra/H r* Was measured by using the outputs from the' I 

appraisal s\stenis, again fortunatc'lv one using standardised forms, I 

l)roduc ing data wbu b ( ould l)c' Lisc'd to create an index of I 

pe'r'ormanc (V Tfu' compan\ was aw'are that the' c]ua!ity of this data was I 
onl\ as good as the pculormancc' apt)raisal system which underpins it, I 
but this IS a separat*' issue lK'\'ond the ('valuation of the* impac t of I 

ln\estors. I 

The comparn undertook an audit of the (ra/n/ng system, C'nabling | 

them to lake an obj(’cti\e \ lew of what thc' company actually did in I 

ti.'rms of tiaining and d('velof)mc'nt. This covered the approach to j 

managing training ((wistc'nc (' of a formal training budgc’t, size, whether it I 
IS all sp(>nt, what it < o\er('d, how' it is dc'cided and by whom) the I 

planning of training (method of training nc'cds analysis, if any, and I 

exislenc v of (ndi\idua! or ( oHec liv(’ training plan), slc'ps to mc'asurc' the I 
(>ff('c ti\enes^ of training <ind the' dt‘livt*r>' ot training. I 

First steps 

Ttie comparn dratlt'd a fiosition p.ifier ori where they were are at the outset: in 
('sst'nce, a lMs(>!me position p.ipi'r, Ihev rc'tognisc'd that if thc'y did not do the. at ' 
the’ time, th('H a relrosfx'c ti\ e <itt('m[>t at a Liter date would lead to some of the 
information not lieing a\ai!al)le Whilst initially sounding like a burc^aucratic process 
It was not Lindui\ st) most n\ tlTi' t»u tors involved wer(' considc'recl when the 
c omparu made its dec ision tc » bre omc- involved in the’ first plac c*. In addition, much 
of th(' information lepuired was mmcIiIv availal)l(*, including information gathc'red 
tor us(' in drawing, up Itie Aceon I’lan little additional information w'ds rec|uireci 
and rv)ne spec its all\ c ( )llf '( N ■( i ,is always, there was a tiad(**off bc'twc'en th(' 
usetLiIness t >t Hie il.it.i and tlie < ms| ot tualK getting it 

l! was ifTter^di’d tii,il \\lie'. itie si lodaid is reac hed, this base ‘line should bf' rc’Visited 
ttir dat I ( ollec t-'d ,r,;aMi Itus en.cbles a Ix'tore <ind after comparison, whic h 
■will indu ate whic h ansis i\e :M;pio\ed o\ei the pc’riod ( aven that (‘arlv (*videnc(' 
sijy;grsis Pial e.ains tmin Inwsints ,iie ac hiev(’(l gradualK', revisiting third timt' 
when Itie standaid has beeo n i, i ,’U aaa '( 1 tor a \\\v m<i\ l>e nec essarv. 
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• bctore-nnd-nfter studies 

• statistical techniques 

• hypc'thetical questioning. 

8.3.1 Control groups 

The use of control groups is the classic design for evaluation. 
Whilst it is not always possible to achieve, it does provide the 
clearest method of measuring the value added by training. 

A control group can either be an individual, sets of individuals, 
organisations or parts of an organisation. The key element of this 
group is that the\- are chosen and then do not receive training, so 
that their behaviour remains constant. The performance of other 
groups who have been trained can then be compared with the 
performance cU the original control group. The measurement 
process is shown in the table below: the impact of the training 
event is the difference between b minus a, and d minus c. That 
is, both target and control group will have changed over time: 
the difference between these different rates of progress is the 
impact of the training. 

Bchire After 

I arj’C’t group a 1) 

Control g'()up ( d 



A crucial factor is deciding who should form the control group. 
In the purest form of experimental design, individuals or groups 
are assigned to control groups on a random basis. If this is not 
possible, control groups should be chosen so that their 
characteristics are similar to those of the target population. 

Th'-re are problems with this form of research design, namely: 

• riiere niav be no extra' pec'ple available to serve as a control 
group since the training may be designed to serv^e everybody. 
In this case, excluding some people from the training (whilst 
it mav be possible) creates additional problems in that senior 
executives mav object to individuals not receiving the 
training and thus lowering their productive capacity. 

• C (intr('l groups must be defined before the start of the action; 
this research design cannot be applied retrospectixely. 

8.3.2 Malching groups 

Matching gioups aie available individuals or groups with 
similar charactei istii, s whi> have not received the benefits of the 
training which can he used as comparators, 
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Wine service training in Forte Hotels 

In 1900, roi t(‘ I lotelb (lc*( i(l(‘(l » > launch a major programme to improve the 
wine service and s('lling skills of their staff. In response to this, The 
IVotessional Wine St‘r\ ice programme - an integrated video, learning guide, 
workl) 0 ()k and standards manual - was develo()ed as j single open and 
tle\il)l(* learning packag(‘ tu d(‘velop trainees' competence. 

In ord(‘r to evaluate llu* eftec ts ut this initiative, the training was introduced 
to (ompanv hoteds and H similar hut independent hotels. The results 
from these were th(»n compared v\ith those of a control group comprising 
n sc'paratcdv selec ted i)ut similar hotels from within the company chain, 
where no iu‘W wine service training v\'as provided. 

The results of tlie training programme were analysed over a 12 month 
f)('riod to gauge reactions to training, practical applications of new skills, 
motivational c hanges in trainees and overall business behaviour. The results 
showed significant business benefits in that the hotels with the training out- 
f)erformed those' in the control group. While the control group experienced 
a decrease of five pc'r ( t'lU m monthly liquor sales eJuring the training period, 
the companv hotc'ls and \Uv independent hotels who used the training 
s( lieme increased thcMr sales l)v sevcMi j)er cent. 

Using this comparative' approach, Forte Motc'Is were able to viemonstrate a 
range of advantages dtaiving from the training programme. These included: 

• MU teased paoduc t sak's 

• higher c wstomer satisf.ic tion 

• imfirovc'd stvitt motivation, and 

• raisc'd iirolc'ssioiuil staiulLirds. 

Th(' existence ot a control grouj) givc's a muc li c learc'r view of the benefits of 
training to tiu* ( )t g.misation. 

[jken trom /a.'-gef [.mploymont Departntent. 



The major diftorence between these and ccmtrol groups is the 
method of selecting the respective groups. Whilst a control 
group is selected c.v mitc, matching groups are chosen ex post. 
They are, therefc^re, not theoretically as rigorous as ccantrol 
groups, but arc‘ possible the most common choice in practice. 

In using matching groups it is important t(^ realise that the aim is 
l(^ take account onlv of tliose factors which could be significant: 
it is ncU possible or c'ssonlial to guard against every source of 
error. 

Before-and-afler studies 

A bc‘(ore-and*after strdv involves observing the bohavic>ur or 
characteristics id gnuips benefiting from training, both before 
and after the’ ewcnl, ami llien examining changes in the variables 
tliat the Iraimng was supposed to affect. Hefore-and-after studies 
lire frevjiu'ntlv uscul in ccHqunetion with control grcnips or 
matching grmips, allbou);h tlu*v have' been used without such 
exteiTial comptirators, 
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Normal practico is lt> establish a baseline studV/ which is an 
examination of the situation before, or at the outset of the 
training. In a few limited cases it may be possible to rely on 
historical data to prt>duce a retrospective base line. 

The main prt)hlem with before-and-after studies (when they are 
used on their own) lies in eliminating extraneous economic 
effects: that is, identifying what would have happened anyway. 



B.3.4 Hypothetical questioning 

I lypolhetical quest iiming involves asking recipients of the 
training what their actions would have been in the absence of 
the training. A ct>mparison between this hypothetical behaviour 
and their actual belun iour will therefore show the effect of the 
training. 

When using this method, the questions need to be carefully 
phrased and the answers interpreted with caution in order to 
avoid contaminated results. Trainees may want to be seen to 
give the 'right' answer and not to appear to have wasted time 
and mone\' on the training. This approach also assumes that the 
traimv actualK' kmnvs what wouki have happened in the 
absence of the training. 

A variant on asking the trainee is to ask the supennsor or 
manager of the trainee tt> ascertain their views on performance 
pre* and post-training. 1 his situation suffers from the same 
problems as those discussed above, but works well in 
coniunclioi'i with a rigorous appraisal system. 



8.4 Which methodology? 

Which research tool is used will usually be driven by practical 
considerations such as the stage of the training's development 
and the qualil\’ of data available. It will usually be the case that a 
mix of research tools may he most appropriate. 

If the evaluation is designed (as it should be) before the training 
starts, then a baseline study can be completed and control or 
matching groups selected. 

A point to bear in mind here is whether the timing is 
appropriate. 1 here may well be cases where evaluation is noi 
going to prt>diu L' a realistic result. lAamples are where the 
trainii'ig aeli\ it\ is tot^ recent or toti distant to allow a proper 
ev aluation, (U* whert‘ lhc‘re ha\ c‘ been one-t>ff events which may 
distiut the external environment, fc^r example a company 
merger. 



l.valiutii'n ^an be i\\\ e\[>ensivc‘ procc’ss but sluuild always be 
si*i‘p in Ib.e context ol the total spc‘iul on the overall training 
programnu*. It, howiver, thi‘ cost of evaluation is a 




- M(MSUrin^; tlu' 1 lte( li\ eness ( )t Iiainin^’ 






47 



disproporticinatL* amount, then an alternative method should be 
considered. What is a reasonable' amount varies, but the most 
expensive tvpes (usually of individuals, which require specially 
designed survevs) involve spending about one per cent of the 
total spent on the training. It should be borne in mind that the 
amount spent on evaluation will vary according to the life cycle 
of a particular course of training; more will be spent at critical 
times, such as during a pilot programme or during changes to 
the training, and less when the training is up and running. 
Overall, however, evaluation costs should be seen as a legitimate 
part of training expenditure. If the costs are separated out into a 
separate budget, then the evaluation process will suffer. This 
will nearlv ahvavs be a false economv. 



8.5 Summary 



Because of the increasing use of, and emphasis on individual 
appraisal, as a management tool and to determine individual 
training needs, the majority of evaluations of the impact of 
training tend to be completed at the level of the individual. 
Whilst useful, most organisations will also want to know the 
impact on the organisation. 

To do this effectiveK , requires an evaluation framework which 
has to be designed in advance. Most frequently used in 
combinations, these can be used at either the individual or 
organisational le\ el. 
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9. Ensure that the Advantages of Training are Maintained 
in the Long Term 



9.1 Introduction 



It 11 h‘ Iniining htis Kvn d success and boih individuals and 
organisdliiMis benefited, then it would seem only sensible to take 
action to ensure that any positive results are kept within the 
organisation tor as lon^ as possible. In particular, it is important 
to t'nsure that the benefits of training are maintained in the long 
term, sinci' it is tjuite possible for employees to improve their 
performance immediately after training and then gradually 
re\ ert to their pre\ ious ways of working. The obverse of this is 
al.so true: training may have negative results (usually where 
there are clashes of organisational culture and where a new 
'learnt' technitjue is not appropriate to the host organisation). In 
these cases training needs to be mi learnt. 

Ihere an* two arwjs we imwe on to di.scuss below^ where the 
research has toimd ex.imples of this use of reinforcement: (i) the 
use* i>f formal rew.ird systen) to ensure that there are positive 
incenti\ es tor em[>lo\ ees in use their new skills and (ii) cemslant 
reassessment. 



Skill-base*el, or aunpc‘tence-based, pay, is a payment system in 
which pa\’ is linke*el dire*ctly io the acquisitiem by employees of 
ne'w skills (usualb in additiem le> a general pay rise). It has been 
ree'eigmse*ei b\ .1 numbe*r of employers as a means te> raise the 
skills base* of a targe*t greuip. It is therefore a suitable system to 
re*ward the* aeeiuisitiem anel use e)f skills ace]uired through 
training, tlu*re*b\ suf^porting the maintenance e>f pe>siti\e 
beha\ iour in the* w e>rkplae e*. 

I he* use* ot pa\ to re'w.irel impro\’eel perfe)rman'\ is a teipical 
issue*. 1 lu'inson ( M notes e\ idence* that suggests tlial between 
.1 halt to tw i^-thiieis o| L. K e^rganisatie^ns may have seime form of 
ineli\ ieiu.il-h. lie'll pe l loi inance related payment (PRP) and that 
this form ol pa\ im nt s\ ste*m is mnv covering a wide range e)f 
e*mplen e*e*s. 1 )e*-pite’ its iiure*asing popularity, semu* diuibt has 
be*e*n east tm the* i*tu*e t i\ e*ne*ss e>t such systems in mei'ting their 

ob)e*e ti\ e*s 



9.2 Use of a formal reward system 
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Computer firm Unis\ s has joineil tlie growing number of organisations using 
competence's to link f)(*rforman( e to reward. An all merit competency-based 
pay scheme covers all 1,400 staff, specialists and managers as part of a long- 
term emplov(‘(' (ievc'lopnu'nl programme aiming to improve the skills and 
befiaviours of (Mn|)lo\(*(*s. 

Unisys defiiu's t()mf)i'tena's as the* ability to perform a job-related task to a 
pre-determined standard', and every employee is assessed against tc^n 
competences, l ive' of th('sc‘ are (ore competences, W'hich Unisys wishes to 
encourage across all ih(‘ org.inisation, and the remaining five are specific to 
each of the four sepai\ite fnncti(')n groups. 

Th(' fio' core conipc'tc'iK es mv : 

• team orientation: woiking willinglv and effc^ctively with others to 
ac( omplisfi goals and to idc’iitifv and resolve problems which may be of 
no dir('ct f)ersonaI inlt'K'sl 

• innovation/creativity: g(*nerating creative and effective solutions to 
business problems and situations. Demonstrating a willingness to try new 
and different approaches. 

• ownership/accountahility; taking rc'sponsibilily for actions and accepting 
assoc latiui risks. 

• quality and customer service orientation: working in partnership with 
customers (both mtc'inal and c‘\ternal) to achieve results which are 
mutualK' l^eiu'Mc lal 

• drive for resulls/initiative: ac tiveK' influencing events rather than 
passivc'K a((ot)ting; s(*eing op|)ortunities and acting upon them; 
originating iic tion. 

All competences an* lalt'd on a s( ak’ of onc‘ to fivc\ with the lowest rating of 
on(‘ indicaiing a ma|oi dev(*!nf)m('nt ne(*d. 1 ivc^ marks the competc^ncy as a 
major strc'ngtii, 

■Annua! salarv nureascs am ( h'termiiK^d l)v a matrix linking competency 
score's to a position on iln' range. One* aim of the new system is to 
(liffc're'ntiate* s[iaipi\ l)etwoeii pe'rtoi mers. 

lake'll tioin // >.S Managevnem Pay Review, july 1994. 



However, the use of such flexible systems gives obvious scope 
for warding indi\'iduals vvhei are perceived to perform better 
than others, and thus may have a role in maintaining 
improvements in pc'rtormance fc^llovving training or 
development. 

Some organisatiiins h.u'e gone further than this and use 
competences to link performance* to reward. A case in point was 
reported in the IDS Management Tay Review (1994), citing the 
example of L'nis\ s (sc'c* hn\), 

9.3 Reassessment of performance 

Oru‘ method id m.imtciinin^ beh.n iour is to constantly re-assess 
perlormanu- to ensiiie that the enhanced behaviour is 
maintained Ihis i.m onl\ he done bv a line manager who is in 
constant contact u itli stall, wlio works with them to ensure that 
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The company, a chain of fast food restaurants, has a well-developed 
competence framework for all the tasks that need to be accomplished 
within its establishments. Lach member of staff is checked on these 
compotonces .ir on gaining them is awarded a 'merit'. The hourly rate for 
the jol) IS (lire( tl\ link(‘d to [hv numl)er of merits that an individual holds. 

To maintain standanis liiu' managers have, as f>art of their job roles, the 
resf)onsil)ilit\ to (oristantK assess the performance of individuals against the 
cornpeteiue (riteria, It an individual is noted as not performing as the 
c()mf)etence fianunsork la\s down, the merit is withdrawn, with the hourly 
rate of \)a\ Ix'ing sul)s(*(]u(*ntly reduced. 



they are behaving as intended. This method is particularly 
effective when linked with a reward system, as noted above. 



9.4 Summary 



I eft to their cuvn devices, individuals may slide back into old 
svstems of working, particularly when this retrograde behaviour 
is tacitly supported by supervisors, colleagues or the w'orking 
en\ iremment. To av oid this requires a combination of positive 
rewards and assessment. 
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10. Presenting the Results 



10.1 Introduction 



The key issue remains the extent to which we can trace these 
elements down to a cost base — if identify the costs and benefits 
(T training in such a way as to allow a financial analysis. 

This is, of course, where the greatest difficulty lies. In the first 
instance, the niajorit\’ of companies do not even fully cost their 
expenditure on training, never mind the benefits that such an 
investment reaps. Below, we outline some of the issues that need 
to be considered. 

10.2 Costing training 

When estimating the elements that could be included in their 
training, emplovers lace a lengthy list. There is a huge variation 
between empkners in what items of expenditure they actually 
include on this list in doing their own calculations. Thus was 
explored bv Dench ( who found that: 

• a relativeK large proportion of employers were unable to 
provide any estimate of training spend at all, and 

• there was no general agreement among employers as to what 
constitutes training expenditure. 

Because of this, wt> do not feel able to prescribe what should be 
included in a costing of training expenditure, but will limit 
ourselves to listing factors which might be included. The choice 
of which of these to clun^se lies between the TIR professional and 
their accountants within each organisation. How this (and 
particularK' fixed cost or overhead items) is then further 
alliKated to individual training events, is a further area for 
consideration, .\ suitable list is gi\en by Newby (19^2) and we 
show this below. 



10.3 Estimating the benefits 

(. onventional .icc(Uinting lochniv|ues are unsuitable for 
providing, tlu' iH’cessar\ lin.incial informatiim abimt human 
resource ssiu's, 
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Items that may be included in a costing of training 
Fixed capital 

• l)uil(ling iind tr\iinin^ rooms 

• fixtures and tittin^',s 

• fixed stMVues ((‘ij aerial sockets, ( ('TV, efc.) 

• e(|uiprnent 

• provision of a liatning rc'source (e^ library) 

• motor veliic les. 

Working capital 

• ( onsumal)le supf)lies 

• f]Kiint(*naiK c' ot e(|uipinent and promise’s 

• mat(*rials used during training 

Administrative and personnel costs of the training function 

• empiovment costs tor training manager, administrative and clerical staff 

• af)[)ortion(Mi costs ot ree* l)usiness rate, heating, lighting, efc. for training 
rooms 

• salaru’s (gross) ot instrut tors traiiu'rs vvlien not training or engaged in 
ci('V(‘lopm(‘nt work. 

Costs of providing instructors/trainers 

• tec‘s tor e\t(‘inal trainiMs (>f trainers' course's 

• f'('c ruitment and selei tton c osts 

• K‘fr('sher and developmental course's 

• salaric's (gross) ot instruc tois trainc'rs vvlien not training or c'ngaged in 
d(‘V(‘lof)m<'nt work 

Costs of training development 

• tees to (‘xtc'rivil c onsultants 

• expc'nses iik urred m |>iodu(ing visual aids printing of course* materials, 
ef( . 

Cost of giving instruclion 

• trav el vind ai ( ( iinmc xlation 

• salaries (gross) ot msiruitois triiineis whc'ii not training or engagc'cl in 
(lev el< ipnx'nt v\ ( ii k 

Taken from Nevvl)v, 



C'arne\ale and Shul/ argue that the traditional balance sheet 
shows organisation's human resources solely as expensc‘s, 
wherc'as lhe\' shoidd be* seen as assets to be optimised. The 
balance sheet also tc'iuls to give a short term view and both these 
factors discourage ijivestment in training. The use of cost benefit 
anal’csis is also inappropriate for training activities because 
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Deriving financial benefits 

('oaley (1994) outliiu's a procedure tor deriving the financial benefits of 
trainitig. Ttie stc*|)S are as loilovss; 

• Costing f)<>of pertormaiu e: individuals with a skill deficiency make 
mistakes, which can then he* accounted for in financial terms. Areas of 
f)Oor performance included c ustomer complaints being inadequately 
dc'alt with, tncom*c t Ic'vel of details on prcjjects, missed deadlines, etc. 
For eac li situation line managc'rs were askc'd for details of direct costs in 
terms of casfi ()aicl out to rectify the situation, lost order** for the 
( ompanv and linu' wastcni. Sa!ar\ levels were recorded in order to derive 
financial costs ot wastcnl time*. 

• Measuring increase in skills: Ix'fore the training event, individuals were 
assessc'd l)\ line managers on a scale of zero (low) to ten, where it was 
agreed that a rating ot s;\ indic ates acceptable', satisfac tory performance. 
Assessment is made post and pre-testing, 

• Areas whc'rc' individuals had ()rogressc'd from below six to six or above 
meant that tasks which wene being done less than satisfactorily before 
were now l)(*ing satisfac torilv, and that costs previously incurreeJ are now 
saved. 

By using this method and comparing it with the costs of the training 
within a large UK organisation, it was estimated that training had raised 
performance to a level which represents a theoretical financial 
contribution to the organisation of £71 million, whilst the costs of that 
training amounted to £7 million. 

Taken from Manai’tvnent iMifung: Cost or Invrstmcnt^ Co^}ley (1994). 



w'hilst the costs arc immediate and apparent, the advantages 
may take some time to be realised. 

However, inevitablv interest will form around putting a 
monetary value on the results of training. An exercise has 
recently put forward a way for this to be done, which is 
explained in the* text box abov e. 



10.4. Presenting the results 

To be ultini.itolv usotul, the results of the evaluation exercise 
need to be fed back to those involved in either policy making or 
developing the training programmes: without these feedback 
loops the 'ovclo' is not complete. It is quite possible that the 
results of vour work will not be appreciated and that elements 
within the organis.ition may resist changes. Simple aw'areness of 
this will alwa\'s help, but turther things to be considered are; 

• Present appropriate intorination to appropriate users. As 
noted abovi', there are different interest groups who will 
want different things from the evaluation. Make sure that the 
right groups are given the right information. 

• Present useful eompa. ■'^ons, proviiling examples of good and 
hai.1 training outcomes, i.igether with some indicators of what 
proiluces gooil and hail outcomes. 
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Using evaluation data 

Reporting on a case siiuK ot a ina)or training programme associated with 
oiu' ot tiu* l)igg(*st non nulitar\ computerisation project^ in Lurope for the 
I )ef)artni(*nt ot Social Securii\. [ asterby and Mackness examined what had 
haf)p(Mie(! to tfic e\ situation (iata and how- it haci been used. It w'as found 
tiiat th(M'e were dittert'nt groups who examined the data for different things, 
suc h that: 

• regional managiMS recc'ived the reports first, skimmed down them, 

checked how thcMr rc'gions were doing against others and passed the 

reports down the line. On the few' occasions that the evaluation reports 
initiated changev tins was sparked off not by the evaluation reports, but 
l)\' comnuMits from tutors and [)articipants. 

• traimns iiacl limited interest in the statistics (although some were 

interested in iiow tiie\ c omf)ar(Hl against other regions). Significant 
(1'1,'inges to tiu'H operations w(»re picked U[) from questionnaires 
completed at tlm end the-* training events. 

• \\)v design tcMin had little time^ for the reports or the ratings from the 
ciuesiionnaires. lor thc'm, the most useful information came from the 
follow 'Uf) int(‘r\i(‘ws and tiu' tutor reports. 

laken from Lasterbv-Smith and Mackness, 1992. 



• Communicate the findings in a non-technical language. The 
process ot measuring the effectiveness of training may be a 
technical process, reporting the findings need not be so. 

• Present the findings at an appropriate time, usually when 
decisions abcuit the training programmes are being made, 
rather than a IcMTg time before or afterwards. 

• l^e honest about the limitations of the evaluation exercise. 
Arguing a case on dubious data damages credibility of all 
who have taken part. 

Thc‘ point o! appropriate information for appropriate users is 
one highlighted bv 1 asterby-Smith and Mackness, sh^^wn in the 
box abew’e, 



10.5 Summary 



Ine\*itably, some locus will be placed on the financial aspects of 
the evaluation procedure: has the training been worthwhile, 
does it show a return to the company? There are no hard and 
fast rule's abenit a ua\ to do this. In this sense, evaluation is as 
much as art as a scic'nce. 1 lowever, the same could be said about 
accoimtanc\; it was ncU for nothing the term 'creative 
accountanc\ ’ w\ts c luned, 

DitU'n'nt c*mpl(wers include' different elements when they total 
the' cc'sts ot Iraimtig. ( )ne approach is as valid as another, hut 
whichc'vc'r itc'ms are included, it must be made clear what is in 
(atui w hat is not) and a reason behind these decisions, 




Measuring tin* 1 ttrn tiV(MU‘ss of 1 Mining 







An estimation of benefits c*in only be made if an estimate of the 
shortfall has been made at the identification of need stage If this 
is the case, then repeating the estimation process should give an 
indication of the value the investment. 

I'inally, consideration must be given to the way in which the 
results t>f the evaluatu'n are fed back. As we have seen in earlier 
chapters, there exists different kinds of information available: 
there are also different stakeholders, operating within their own 
time frames. 
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11. Conclusions and Summary 



The tmining evaluation literature incorporates a wide variety of 
approaches, which range from highly theoretical models to 
practical manuals and texts. While the adoption of a well 
thought out model can avoid money being wasted in training, it 
is clear that time, resources and money for evaluation are 
limited. In addition, .t is clear that there remain a good many 
i^bstacles facing an evaluation of training. The Industrial Society 
notes that over a half of organisations mention that the difficulty 
lies in establishing measurable results, which indicates a lack of 
knowledge of the appropriate techniques. Half mention a lack of 
time, with a third noting a lack of line manager support. 

In practice, therefore, evaluation techniques need to be clearly 
thought out but also manageable in their levels of complexity 
and sophistication. The gap between theory and practice needs 
to be narrowed and this review has attempted to highlight some 
basic issues in the debate. 



l or those iinolved in the practice of delivering training and 
interested in evaluating the outcomes of their activities, the 
pertinent point is more than likely to be 'where do I start?'. The 
answer to this is that they must start at the most obvious place: 
at the beginning, which, in this case, is the identification of 
training needs. This has to be the starting point, for without it 
the rest of the stages of the training cycle are based on poor 
foundations. It must be ensured, however, that it does not also 
become the final stage. A training department, which promises 
to deliver an indicator of value for money of its activities and 
only gets as tar as identifying needs (which undoubtedly will be 
seen as a plo\ to .Utract more resource and investment) will not 
enhance its u'piitation throughout the evganisation. 



It is likeK that the training manager will not be able to evaluate 
all the training that takes place within the organisation 
according to the s\ stems described in this report. There is a need 
tor prioritisation: which happens most frequently, which is most 
iH)stIv, but perhaps most importantly, which training and 
ileveh'pment is most important for the organisation. 
Determining priorities will allow some evaluation to take place 
whilst otluM* will sta\’ undone. I lowevcT, it will provide the 
training m«m«)gc‘r with the most impcMtant of tools, clarity. It will 
allow tin* training inan<igt*r to l)i* clear about why they are doing 
things ami inU others 
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